97 research outputs found
Stopping Rules for Gradient Methods for Non-Convex Problems with Additive Noise in Gradient
We study the gradient method under the assumption that an additively inexact
gradient is available for, generally speaking, non-convex problems. The
non-convexity of the objective function, as well as the use of an inexactness
specified gradient at iterations, can lead to various problems. For example,
the trajectory of the gradient method may be far enough away from the starting
point. On the other hand, the unbounded removal of the trajectory of the
gradient method in the presence of noise can lead to the removal of the
trajectory of the method from the desired exact solution. The results of
investigating the behavior of the trajectory of the gradient method are
obtained under the assumption of the inexactness of the gradient and the
condition of gradient dominance. It is well known that such a condition is
valid for many important non-convex problems. Moreover, it leads to good
complexity guarantees for the gradient method. A rule of early stopping of the
gradient method is proposed. Firstly, it guarantees achieving an acceptable
quality of the exit point of the method in terms of the function. Secondly, the
stopping rule ensures a fairly moderate distance of this point from the chosen
initial position. In addition to the gradient method with a constant step, its
variant with adaptive step size is also investigated in detail, which makes it
possible to apply the developed technique in the case of an unknown Lipschitz
constant for the gradient. Some computational experiments have been carried out
which demonstrate effectiveness of the proposed stopping rule for the
investigated gradient methods
Accuracy guaranties for recovery of block-sparse signals
We introduce a general framework to handle structured models (sparse and
block-sparse with possibly overlapping blocks). We discuss new methods for
their recovery from incomplete observation, corrupted with deterministic and
stochastic noise, using block- regularization. While the current theory
provides promising bounds for the recovery errors under a number of different,
yet mostly hard to verify conditions, our emphasis is on verifiable conditions
on the problem parameters (sensing matrix and the block structure) which
guarantee accurate recovery. Verifiability of our conditions not only leads to
efficiently computable bounds for the recovery error but also allows us to
optimize these error bounds with respect to the method parameters, and
therefore construct estimators with improved statistical properties. To justify
our approach, we also provide an oracle inequality, which links the properties
of the proposed recovery algorithms and the best estimation performance.
Furthermore, utilizing these verifiable conditions, we develop a
computationally cheap alternative to block- minimization, the
non-Euclidean Block Matching Pursuit algorithm. We close by presenting a
numerical study to investigate the effect of different block regularizations
and demonstrate the performance of the proposed recoveries.Comment: Published in at http://dx.doi.org/10.1214/12-AOS1057 the Annals of
Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical
Statistics (http://www.imstat.org
Smooth Monotone Stochastic Variational Inequalities and Saddle Point Problems -- Survey
This paper is a survey of methods for solving smooth (strongly) monotone
stochastic variational inequalities. To begin with, we give the deterministic
foundation from which the stochastic methods eventually evolved. Then we review
methods for the general stochastic formulation, and look at the finite sum
setup. The last parts of the paper are devoted to various recent (not
necessarily stochastic) advances in algorithms for variational inequalities.Comment: 12 page
On existence and stability of equilibria of linear time-invariant systems With constant power loads
The problem of existence and stability of equilibria of linear systems with constant power loads is addressed in this paper. First, we correct an unfortunate mistake in our recent paper [10] pertaining to the sufficiency of the condition for existence of equilibria in multiport systems given there. Second, we give two necessary conditions for existence of equilibria. The first one is a simple linear matrix inequality hence it can be easily verified with existing software. Third, we prove that the latter condition is also sufficient if a set defined by the problem data is convex, which is the case for single and two-port systems. Finally, sufficient conditions for stability and instability for a given equilibrium point are given. The results are illustrated with two benchmark examples.Postprint (author's final draft
- …